Self-adaptive deep neural network: Numerical approximation to functions and PDEs

Designing an optimal deep neural network for a given task is important and challenging in many machine learning applications. To address this issue, we introduce self-adaptive algorithm: the adaptive enhancement (ANE) method, written as loops of formtrain→estimate→enhance. Starting with small two-layer (NN), step train to solve optimization problem at current NN; estimate compute posteriori estimator/indicators using solution enhance add new neurons NN. Novel strategies based on computed are developed paper determine how when layer should be added The ANE method provides natural process obtaining good initialization training addition, advanced procedure initialize newly better approximation. We demonstrate that can automatically design nearly minimal NN functions exhibiting sharp transitional layers well discontinuous solutions hyperbolic partial differential equations.